EigenSense: Saving User Effort with Active Metric Learning
نویسندگان
چکیده
Research in interactive machine learning has shown the effectiveness of live, human interaction with machine learning algorithms in many applications. Metric learning is a common type of algorithm employed in this context, using feedback from users to learn a distance metric over the data that encapsulates their own understanding. Less progress has been made on helping users decide which data to examine for potential feedback. Systems may make suggestions for grouping items, or may propose constraints to the user, generally by focusing on fixing areas of uncertainty in the model. For this work-in-progress, we propose an active learning approach, aimed at an interactive machine learning context, that tries to minimize user effort by directly estimating the impact on the model of potential inputs, and querying users accordingly. With EigenSense, we use eigenvector sensitivity in the pairwise distance matrix induced by a distance metric over the data to estimate how much a given user input might affect the metric. We evaluate the technique by comparing the output points it proposes for user consideration against what an oracle would like to choose as inputs.
منابع مشابه
Active learning with minimum expected error for spoken language understanding
Active learning is a strategy to minimize the annotation effort required to train statistical models, such as a statistical classifier used for natural language call routing or user intent classification. Most variants of active learning are “certainty-based;” they typically select, for human labeling, samples that are most likely to be mis-classified by automatic procedures. This approach, whi...
متن کاملActive Metric Learning from Relative Comparisons
This work focuses on active learning of distance metrics from relative comparison information. A relative comparison specifies, for a data point triplet (xi, xj , xk), that instance xi is more similar to xj than to xk. Such constraints, when available, have been shown to be useful toward defining appropriate distance metrics. In real-world applications, acquiring constraints often require consi...
متن کاملActive learning: a step towards automating medical concept extraction
OBJECTIVE This paper presents an automatic, active learning-based system for the extraction of medical concepts from clinical free-text reports. Specifically, (1) the contribution of active learning in reducing the annotation effort and (2) the robustness of incremental active learning framework across different selection criteria and data sets are determined. MATERIALS AND METHODS The compar...
متن کاملActive Learning for Classifying Phone Sequences from Unsupervised Phonotactic Models
This paper describes an application of active learning methods to the classification of phone strings recognized using unsupervised phonotactic models. The only training data required for classification using these recognition methods is assigning class labels to the audio files. The work described here demonstrates that substantial savings in this effort can be obtained by actively selecting e...
متن کاملCost-sensitive active learning for computer-assisted translation
Machine translation technology is not perfect. To be successfully embedded in real-world applications, it must compensate for its imperfections by interacting intelligently with the user within a computer-assisted translation framework. The interactive-predictive paradigm, where both a statistical translation model and a human expert collaborate to generate the translation, has been shown to be...
متن کامل